Dynamic Routing between Two Queues with Unreliable Servers

We examine the problem of how best to route jobs among two queues whose servers are subject to breakdowns and repairs. The optimal routing policy is computed by modelling the system as a discrete-time, finite-state Markov decision process and solving the resulting dynamic programming equations either iteratively or by applying the ‘policy improvement’ algorithm. In a series of numerical experiments, the performance of various heuristic policies is compared with that of the optimal policy. A particular heuristic is shown to be close to optimal over a wide range of parameters.